Skip to content

Conversation

@cnanakos
Copy link
Contributor

No description provided.

gmega and others added 29 commits September 30, 2025 14:52
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Giuliano Mega <[email protected]>
…or getBlocks; limit memory usage for fetchBatched when used as prefetcher
Use probabilistic distribution based on peer quality scores, giving all peers
opportunity while favoring better-performing ones. Selection probability is
inversely proportional to score.

Part of #974
- Reduce DefaultFetchBatch to prevent blockData explosion
- Add chunked onBatch processing to prevent memory accumulation
- Implement memory buffer reuse to reduce GC allocation churn
- Fix short-circuit evaluation

Part of #974
This prevents unbounded peer accumulation while maintaining redundancy.
When peer count exceeds the maximum, the least active peers are removed from
tracking to free resources.

Part of #974
Achieves significant memory reduction with equivalent network
performance. The reduced batch size prevents memory pressure
while preserving transfer efficiency, improving overall system
resource utilization.

Part of #974
- Add codex_block_exchange_discovery_requests_total counter to track peer
  discovery frequency
- Add codex_block_exchange_peer_timeouts_total counter to monitor peer
  reliability issues
- Add codex_block_exchange_requests_failed_total counter to track request
  failure rates

Part of #974
This prevents peers from becoming permanently invisible to block discovery when
they fail to respond to WantHave requests.

Part of #974
Returns false on duplicate marking attempts instead of logging errors,
eliminating duplicate marking loop in blockPresenceHandler and
preventing duplicate block requests across concurrent flows.

Part of #974
Part of #974

Signed-off-by: Chrysostomos Nanakos <[email protected]>
Part of #974

Signed-off-by: Chrysostomos Nanakos <[email protected]>
@cnanakos cnanakos force-pushed the feat/simple-optimizations-rebase branch from 46cc4ad to 372151e Compare October 1, 2025 17:03
@cnanakos cnanakos force-pushed the feat/simple-optimizations-rebase branch 22 times, most recently from 85eb620 to 5cf5f00 Compare October 22, 2025 13:40
@cnanakos cnanakos force-pushed the feat/simple-optimizations-rebase branch 2 times, most recently from 7bddc94 to f8f2e40 Compare October 23, 2025 14:25
Part of #974

Signed-off-by: Chrysostomos Nanakos <[email protected]>
@cnanakos cnanakos force-pushed the feat/simple-optimizations-rebase branch from f8f2e40 to b8c2393 Compare October 24, 2025 08:55
Implements delta-based WantList updates to reduce network traffic during
block exchange. Only sends newly added blocks instead of resending the
entire WantList on every refresh.

Also some network related fixes:

- Add TCP_NODELAY flag to prevent Nagle's algorithm delays
- Clear sendConn on stream reset to allow garbage collection
- Improve error handling in NetworkPeer.send()

Part of #974

Signed-off-by: Chrysostomos Nanakos <[email protected]>
Now fetchBatched maintains a sliding window of batchSize blocks in-flight.
When 50% complete, adds next chunk (batchSize/2) to maintain constant window
size. This ensures blocks are already pending or have been fetched when
StoreStream needs them.

Signed-off-by: Chrysostomos Nanakos <[email protected]>
@cnanakos cnanakos force-pushed the feat/simple-optimizations-rebase branch from b8c2393 to c09a247 Compare October 24, 2025 13:30
Signed-off-by: Chrysostomos Nanakos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants